{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0",
   "metadata": {},
   "source": [
    "# Backtest: Bybit OrderBook data\n",
    "\n",
    "Tutorial for [NautilusTrader](https://nautilustrader.io/docs/latest/) a high-performance algorithmic trading platform and event driven backtester.\n",
    "\n",
    "[View source on GitHub](https://github.com/nautechsystems/nautilus_trader/blob/develop/docs/tutorials/backtest_bybit_orderbook.ipynb).\n",
    "\n",
    ":::info\n",
    "We are currently working on this tutorial.\n",
    ":::"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1",
   "metadata": {},
   "source": [
    "## Overview\n",
    "\n",
    "This tutorial runs through how to set up the data catalog and a `BacktestNode` to backtest an `OrderBookImbalance` strategy or order book data. This example requires order book depth data provided by Bybit.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2",
   "metadata": {},
   "source": [
    "## Prerequisites\n",
    "\n",
    "- Python 3.12+ installed\n",
    "- [JupyterLab](https://jupyter.org/) or similar installed (`uv pip install jupyterlab`)\n",
    "- [NautilusTrader](https://pypi.org/project/nautilus_trader/) latest release installed (`uv pip install nautilus_trader`)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3",
   "metadata": {},
   "source": [
    "## Imports\n",
    "\n",
    "We'll start with all of our imports for the remainder of this guide:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import shutil\n",
    "from decimal import Decimal\n",
    "from pathlib import Path\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "from nautilus_trader.adapters.bybit.loaders import BybitOrderBookDeltaDataLoader\n",
    "from nautilus_trader.backtest.node import BacktestDataConfig\n",
    "from nautilus_trader.backtest.node import BacktestEngineConfig\n",
    "from nautilus_trader.backtest.node import BacktestNode\n",
    "from nautilus_trader.backtest.node import BacktestRunConfig\n",
    "from nautilus_trader.backtest.node import BacktestVenueConfig\n",
    "from nautilus_trader.config import ImportableStrategyConfig\n",
    "from nautilus_trader.config import LoggingConfig\n",
    "from nautilus_trader.core.datetime import dt_to_unix_nanos\n",
    "from nautilus_trader.model import OrderBookDelta\n",
    "from nautilus_trader.persistence.catalog import ParquetDataCatalog\n",
    "from nautilus_trader.persistence.wranglers import OrderBookDeltaDataWrangler\n",
    "from nautilus_trader.test_kit.providers import TestInstrumentProvider"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5",
   "metadata": {},
   "source": [
    "## Loading data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Path to your data directory, using user /Downloads as an example\n",
    "DATA_DIR = \"~/Downloads\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_path = Path(DATA_DIR).expanduser() / \"Data\" / \"Bybit\"\n",
    "raw_files = list(data_path.iterdir())\n",
    "assert raw_files, f\"Unable to find any histdata files in directory {data_path}\"\n",
    "raw_files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We'll use orderbook depth 500 data provided by Bybit with limit of 1000000 rows\n",
    "path_update = data_path / \"2024-12-01_XRPUSDT_ob500.data.zip\"\n",
    "nrows = 1_000_000\n",
    "df_raw = BybitOrderBookDeltaDataLoader.load(path_update, nrows=nrows)\n",
    "df_raw.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9",
   "metadata": {},
   "source": [
    "### Process deltas using a wrangler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10",
   "metadata": {},
   "outputs": [],
   "source": [
    "XRPUSDT_BYBIT = TestInstrumentProvider.xrpusdt_linear_bybit()\n",
    "wrangler = OrderBookDeltaDataWrangler(XRPUSDT_BYBIT)\n",
    "\n",
    "deltas = wrangler.process(df_raw)\n",
    "deltas.sort(key=lambda x: x.ts_init)  # Ensure data is non-decreasing by `ts_init`\n",
    "deltas[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11",
   "metadata": {},
   "source": [
    "### Set up data catalog"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12",
   "metadata": {},
   "outputs": [],
   "source": [
    "CATALOG_PATH = os.getcwd() + \"/catalog\"\n",
    "\n",
    "# Clear if it already exists, then create fresh\n",
    "if os.path.exists(CATALOG_PATH):\n",
    "    shutil.rmtree(CATALOG_PATH)\n",
    "os.mkdir(CATALOG_PATH)\n",
    "\n",
    "# Create a catalog instance\n",
    "catalog = ParquetDataCatalog(CATALOG_PATH)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "13",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write instrument and ticks to catalog\n",
    "catalog.write_data([XRPUSDT_BYBIT])\n",
    "catalog.write_data(deltas)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "14",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Confirm the instrument was written\n",
    "catalog.instruments()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "15",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Explore the available data in the catalog\n",
    "start = dt_to_unix_nanos(pd.Timestamp(\"2022-11-01\", tz=\"UTC\"))\n",
    "end =  dt_to_unix_nanos(pd.Timestamp(\"2022-11-04\", tz=\"UTC\"))\n",
    "\n",
    "deltas = catalog.order_book_deltas(start=start, end=end)\n",
    "print(len(deltas))\n",
    "deltas[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16",
   "metadata": {},
   "source": [
    "## Configure backtest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "17",
   "metadata": {},
   "outputs": [],
   "source": [
    "instrument = catalog.instruments()[0]\n",
    "book_type = \"L2_MBP\"  # Ensure data book type matches venue book type\n",
    "\n",
    "data_configs = [BacktestDataConfig(\n",
    "        catalog_path=CATALOG_PATH,\n",
    "        data_cls=OrderBookDelta,\n",
    "        instrument_id=instrument.id,\n",
    "        # start_time=start,  # Run across all data\n",
    "        # end_time=end,  # Run across all data\n",
    "    )\n",
    "]\n",
    "\n",
    "venues_configs = [\n",
    "    BacktestVenueConfig(\n",
    "        name=\"BYBIT\",\n",
    "        oms_type=\"NETTING\",\n",
    "        account_type=\"CASH\",\n",
    "        base_currency=None,\n",
    "        starting_balances=[\"200000 XRP\", \"100000 USDT\"],\n",
    "        book_type=book_type,  # <-- Venues book type\n",
    "    )\n",
    "]\n",
    "\n",
    "strategies = [\n",
    "    ImportableStrategyConfig(\n",
    "        strategy_path=\"nautilus_trader.examples.strategies.orderbook_imbalance:OrderBookImbalance\",\n",
    "        config_path=\"nautilus_trader.examples.strategies.orderbook_imbalance:OrderBookImbalanceConfig\",\n",
    "        config={\n",
    "            \"instrument_id\": instrument.id,\n",
    "            \"book_type\": book_type,\n",
    "            \"max_trade_size\": Decimal(\"1.000\"),\n",
    "            \"min_seconds_between_triggers\": 1.0,\n",
    "        },\n",
    "    ),\n",
    "]\n",
    "\n",
    "# NautilusTrader currently exceeds the rate limit for Jupyter notebook logging (stdout output),\n",
    "# this is why the `log_level` is set to \"ERROR\". If you lower this level to see\n",
    "# more logging then the notebook will hang during cell execution. A fix is currently\n",
    "# being investigated which involves either raising the configured rate limits for\n",
    "# Jupyter, or throttling the log flushing from Nautilus.\n",
    "# https://github.com/jupyterlab/jupyterlab/issues/12845\n",
    "# https://github.com/deshaw/jupyterlab-limit-output\n",
    "config = BacktestRunConfig(\n",
    "    engine=BacktestEngineConfig(\n",
    "        strategies=strategies,\n",
    "        logging=LoggingConfig(log_level=\"ERROR\"),\n",
    "    ),\n",
    "    data=data_configs,\n",
    "    venues=venues_configs,\n",
    ")\n",
    "\n",
    "config"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18",
   "metadata": {},
   "source": [
    "## Run the backtest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19",
   "metadata": {},
   "outputs": [],
   "source": [
    "node = BacktestNode(configs=[config])\n",
    "\n",
    "result = node.run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "20",
   "metadata": {},
   "outputs": [],
   "source": [
    "result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "21",
   "metadata": {},
   "outputs": [],
   "source": [
    "from nautilus_trader.backtest.engine import BacktestEngine\n",
    "from nautilus_trader.model import Venue\n",
    "\n",
    "\n",
    "engine: BacktestEngine = node.get_engine(config.id)\n",
    "\n",
    "engine.trader.generate_order_fills_report()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "22",
   "metadata": {},
   "outputs": [],
   "source": [
    "engine.trader.generate_positions_report()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "23",
   "metadata": {},
   "outputs": [],
   "source": [
    "engine.trader.generate_account_report(Venue(\"BYBIT\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "24",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
